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Introduction: The Onboard Autonomous 

Science Investigation System (OASIS) evaluates 
geologic data gathered by a planetary rover [1]. 
This analysis is used to prioritize the data for 
transmission, so that the data with the highest 
science value is transmitted to Earth. In addition, 
the onboard analysis results are used to identify 
science opportunities. A planning and scheduling 
component of the system enables the rover to 
take advantage of the identified science 
opportunity. OASIS is a NASA-funded research 
project that is currently being tested on the FIDO 
rover at JPL for the use on future missions. 

OASIS currently works in a closed loop fashion 
with onboard control software (e.g., navigation 
and vision) and has the ability to autonomously 
perform the following sequence of steps: analyze 
gray scale images to find rocks, extract the 
properties of the rocks, identify rocks of interest, 
retask the rover to take additional imagery of the 
identified target and then allow the rover to 
continue on its original mission. We have 
conducted a number of tests of the combined 
system and individual components. We describe 
results for the system in detecting and reacting to 
a science alert, as well as the reliability of the 
rock finder and run time performance. 

Integration with CLARAty: To test OASIS with 
rover hardware, our approach has been to 
integrate with the CLARAty robotic architecture 
[2]. CLARAty is a unified and reusable robotic 
software architecture that simplifies the 
integration of new technologies onto different 
robotic platforms. For this testing, the OASIS 
rockfinder and planning and scheduling software 
were directly integrated with CLARAty and 
handled interaction with other key CLARAty 
elements that were required to run with rover 
hardware. Other pieces of OASIS (such as 
feature extraction) are planned for the future. 

Science Alert: One of the modes of operation for 
OASIS is to search for specific targets during a 
traverse. These are targets that the science team 
has specified as important if encountered by the 
rover. When one of these targets is found, it is 
identified as a new science opportunity and a 
“science alert” is sent to the planning and 
scheduling component of OASIS. After 


reviewing the rover’s current operational status 
to ensure that it has enough resources to 
complete its traverse and act on the new science 
opportunity, OASIS changes the command 
sequence of the rover. 

When a science alert occurs, the rover is 
instructed to stop its current traverse, locate the 
rock (target) that triggered the science alert, and 
take additional data (e.g., color image, closer 
grayscale image, spectrometer reading) on that 
rock. Once it has completed this additional 
measurement, the rover reverts back to its 
original plan and continues on its traverse. 

We have performed a series of tests in the JPL 
Mars Yard using the FIDO rover to demonstrate 
the performance of the science alert capability of 
OASIS. Here we report results from 10 of our 
most recent runs. These tests were conducted on 
four different (non-consecutive) days. The runs 
had an average of 13 images. The criteria 
specified was based on albedo - identify rocks 
that were white. An example is shown in Figure 
1. The goal was to detect targets if they are 
within approximately 3m of the forward field of 
view of the rover as it proceeds along the 
traverse. Each run had between one and seven 
targets that the rover encountered within the goal 
detection range. Over the 10 traverse test runs, 
this consisted of 40 total target rocks. 36 of 
these targets were detected (85% detection rate). 
There were no false alarms during any of the ten 
runs. 

The 85% detection rate indicates how many of 
the targets were identified at some time over the 
course of the traverse. During a traverse, 
however, a target may be imaged multiple times. 
Over the 10 runs, there were 82 instances of the 
targets appearing in an image within the 
detection range. Individual targets were imaged 
between 1 and 6 times. Of the six targets that 
were missed, three of these appeared in a single 
image and three appeared in two images. No 
target that appeared in more than two images 
was missed. 

Rock finder performance: To identify targets, 
the rocks are first detected and then properties of 
the rocks extracted. This relies heavily on the 
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performance of initially finding the rocks in an 
image. Further, automated detection of rocks in 
Mars rover images is also a useful ground-based 
endeavour to alleviate the tedious task of manual 
effort necessary for rock density analyses as in 
[3], In addition to the system testing for science 
alert described above, we have tested the 
performance of the rock finder [4] individually. 
The performance of the rock finding algorithm is 
measured by comparing the rocks identified in an 
image by the algorithm the rocks manually 
identified in the image. Two types of 
comparison have been used in scoring the labels 
returned by the rock finder. Candidate rocks 
produced by the rock finder are compared with 
the hand labeled rocks based on either the 
distance between the centers or the number of 
overlapping pixels. Precision and recall were 
measured. Scoring, or the matching of detected 
rocks to hand labeled rocks, establishes the 
number of True Positive (TP), False Positive 
(FP), and False Negative (FN) detections. 

The performance metrics of Precision is 
calculated for each of the two scoring methods. 
Precision is the fraction of total rocks detected 
that appear in the hand labeled set, Precision = 
TP / (TP + FP). Precision captures the degree to 
which the detector finds only rocks; precision is 
the percentage of objects identified by the rock 
finder as rocks that are truly rocks. A high 
precision directly corresponds to a low false 
alarm rate. 

Precision numbers are given for a total 110 of 
FIDO hazcam images and 2942 total rocks 
analyzed. The two scoring methods show 
similar results. The average precision per image 
using the center matching method was 89%, 
while the average precision using the overlap 
method was 87%. 83 images had higher than 
90% precision. We are currently evaluating the 
precision as a function of distance to the rock. 

OASIS component run times on FIDO: In 
addition to testing the rate of detecting targets, as 
part of our testing, we gathered preliminary 
statistics on the run time of key OASIS 
components. While we have not spent significant 
time optimizing the performance of these 


components, the numbers provide a general idea 
of current performance and provide a reference 
to track future improvements. In a representative 
run, the OASIS rockfinder processed 1 1 hazcam 
images resulting in 5 science alerts being sent to 
the planner. The rockfinder was run on a 233 
Mhz Pentium processor running VxWorks 5.5 
with 128 MB of RAM. Rockfinder took an 
average of 53 seconds to process each image and 
found about 9 rocks per image with a total of 103 
rocks being identified for the 11 images. The 
OASIS feature extraction component was run on 
a 930 Mhz Pentium processor running Linux 2.4 
with 256 MB of RAM. For the current tests, 
albedo and shape information were extracted for 
each rock. For the 1 1 images processed, feature 
extraction averaged 0.5 seconds per image. The 
planning and scheduling component ran on a 2.5 
GFlz Pentium processor running Linux 2.4 with 
1 GB of RAM. In handling the 5 science alerts, 
the planner spent an average of 6 seconds 
generating a plan for each alert. 
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Figure 1. FIDO image. Target rock is 
indicated by an arrow. This rock was 
successfully identified. 




